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ABSTRACT 

This paper examines Virginia's campus-based 
assessment process in the areas of curriculum improvement, academic 
management, accountability, and reporting. While there is 
considerable variation among institutions, assessment has 
demonstrated a moderate to substantial impact on most campuses in the 
area of curriculum improvement. There is evidence that the process 
has resulted in the redesign or modification of major programs and a 
recent trend toward combining assessment with program review. There 
is a concern, however, that the assessment process used at the 
management level to make negative decisions (e.g., program 
elimination) may jeopardize the objectivity with which faculty 
approach the task. The assessment process, it is argued, must not 
only address institutional accountability in terms of efficiency but 
must also contain indicators that reveal improvements in the quality 
of the outcomes of that education. Only with this double approach can 
higher education institution: convince the general public of the 
value of their investment in higher education. Finally, a 
recommendation is made for changing the assessment process so that 
assessment reporting would first involve the submission of an 
assessment activity summary for review, followed by a presentation by 
appropriate institutional officers during which reactions to the 
assessment would be given, follow"up information requested, and 
advice given for program improvement. (GLR) 
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Now that the third biennial assessment reports have just 
been reviewed, it may be time to assess assessment in Virginia — 
what it does well, what it does not do or do well, and how those 
things line up with higher education's needs at a time when it is 
adjusting to a new place in a new scheme of things. The Council 
itself felt the need for such a review and so asked the staff to 
do a session on assessment at its fall retreat, Peter Ewell and 
I 3ed the discussion that day, both of us fresh from the review 
of the senior institutions' reports, 

Peter's conclusion was that campus-based assessment in 
Virginia has reached its maturity. By this he didn't mean to say 
that very good and innovative work doesn't lie ahead — quite the 
contrary. Some programs in Virginia are, in his view, doing the 
most cutting-edge work in the country. But programs have pretty 
much found their niches on campus, their campus-specific natures, 
and their relative positions in the state. So now state-level 
focus needs to be less on quality control of and technical 
assistance to assessment on campus, although we will continue to 
provide those services, and more on the role mature assessment 
programs can play as we and the institutions adjust to changing 
and harsh fiscal realities, as well as changing and sometimes 
even harsher public perceptions. And the assessment leadership 
in the state, which is pretty much gathered together in this 
room, also needs to think about changes in the reporting 
relationship between the Council and the institutions, I would 
like to address both of those issues in my remarks today, 

I like this tradition of giving a speech on assessment once 
a year, because it gives me an opportunity to look back over the 
years, review what I've said before, and see themes developing. 
The theme that began to emerge in 1990 was "accountability," 
Assessment in Virginia, I'm sure you remember, was originally 
conceived to meet the dual goals of improvement and 
accountability, which, in the late eighties, were seen as 
complementary goals that could be served by the same mechanisms. 
Of the two, improvement took priority, which influenced a number 
of decisions about the shape of the program: the renunciation of 
state-wide testing for not just campus-specific but program- 
specific assessment, designed and owned by faculty and structured 
to make cross-institutional or even cross-program comparisons 
impossible. 
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So h^r hcx:^ the prograxti worked to effect improvement in the 
curriculuBv 'Ihere is considerable variation among institutions, 
but genera.Wy, cn most campuses it seems to have had moderate to 
substantial \mpcict. That impact has been most direct at the 
program level,- where it has, above all, fostered a new habit of 
mind. As one c.ep \rtment put it, "Because of the faculty's 
increased involves -^nt in assessment . . . the department is 
showing increas' :\g ^:igns of seeing the need for studying the 
major as an ovei 11 entity and not simply as a random collection 
of individual of: -ri;:c;s taught by, and under the control of, 
individual instruc.o^ But assessment has also affected the 

content of curricu^v: there are many examples of major programs 
modified or complete redesigned, honors and remedial programs 
demonstrably improvec ^ The most recent development is the 
tendency of departments to combine assessment with program 
review. This is good insofar as it places assessment in a larger 
context of evaluation. T.e danger is that in some cases, direct 
performance measures c-'.^xpooar in the new, combined process. 

Assessment in general tlucation continues to reveal the lack 
of intellectual coherence ..n oost programs. As one institutional 
report put it, "The distribution requirements are not organized 
around a set of pedagogical pr ncriples ; they are designed to 
ensure that each relevant dep:^r\ ^r." nt is represented in the 
requirements." On the other r?ard, the need to assess the 
general-education program has i to good conversations and 
widespread curricular revision. 'yne new conversations about 
general education on many campusor^ ax^e a fine example of how 
thinking in learning-outcomes ter.:;' •:an shift the grounds of 
discussion from turf to intention. But there is a disturbing 
pattern of not building assessment iiito the new programs, so it 
remains to be seen whether the new curricul-^ will be any more 
susceptible to assessment than the old onev were. 

It may be that we are on the wrong track here, that what we 
should be looking at is the general maturatir.n of some few 
fundamental intellectual skills though the tindergraduate years. 
As I mentioned two years ago, we are waiting for a federal test 
that is being developed for the National Goa3 : Panel. But even if 
faculty were to administer such a test and fi;:.' tbat students on 
their campus did not show progress in these axce^s, they would 
still have the challenge of translating that knowledcfe into 
curricular changes that make a difference. We need to be 
prepared not only to describe our results but to understand by 
what practices we produce and can improve them. 

I am convinced that we cannot give up on this arc-:--^ of th^-^ 
curriculum — it does, after all. represent at least half of the 
coursework students do and hence a considerable public 
investment. And employers and the public, if they have any 



dissatisfaction with higher education's product, complain about 
graduates' general intellectual, rather than their technical, 
skills — for instance, that they can't communicate or solve 
problems well enough. The Big 6 accounting firms' report on 
accounting education, as one example, says that we are turning 
out technicians rather than "learning machines." I think we 
should welcome the public's desire to equate quality in higher 
education with the fostering of such skills in our graduates, 
because it constitutes a counter-weight to the tendency to equate 
quality with the academic preparation of our incoming students or 
the research reputations of our faculty. But we need to figure 
out how to nail this piece of jello to the wall. 

We had not foreseen that campuses would begin to assess the 
affective development of students. On many campuses, real 
attention is being paid to how students experience campus life, 
in all its elements, and a move is visible to systematically 
assess students-affairs activities. This is a helpful broadening 
of attention. I have made the argument with the chief student 
affairs officers that assessment of their contribution to 
education in the broadest sense may be necessary to the survival 
of their part of the enterprise, since many of these programs 
have suffered serious frostbite in recent years as the blood has 
rushed to the heart. However, in this area there has been a 
tendency to rely exclusively on student-satisfaction measures or 
on counting numbers of students served rather than trying to 
determine how a given activity contributes to students' 
education. 

Some very exciting work is being done on the border between 
cognitive and affective development. The reason this is so 
exciting is that some of the skills that employers are looking 
for in students involve an affective dimension and may best be 
learned outside the classroom: e.g., the capacity to work in 
groups from student clubs and government or the capacity to work 
in an increasingly diverse workplace through dormitory programs. 
Even more fundamentally, an acknowledgement of the fact that 
education happens outside as well as inside the classroom is the 
first step in moving away from the credit-f or-contact model to a 
system of performance-based credentialling. Some institutions are 
beginning to pull down the Berlin Wall separating the curriculum 
and extra-curriculum » A recent initiative at James Madison 
University suggests the role assessment can play in this process: 
the assessment office there proposes to assess students' oral 
communications skills and le^ive open the question of where they 
pick up those skills, whether it be in a public speaking class or 
through the debate team. And the EPISTAR project at Virginia 
Military Institute suggests another way that boundaries can be 
crossed: the assessment office provides advisors with an 
electronic portrait of their advisees, made up of both cognitive 
and affective measures. They can then intervene appropriately 
with high-risk students. But these are the cutting edge efforts 
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in the -state — as yet, most campuses are just entering this 
realm and are, at best, at the "too much [unconnected] data, too 
little information" stage described by NCHEMS. 

Academic management 

Change on a campus can happen at three levels: at the level 
of the program, the dean, and the provost or president. As I 
mentioned, we have seen numerous examples of changes at the 
program level. There are also some institutions where 
assessment-driven change is happening at the deans' level. At 
the College of William and Mary, Radford University, and George 
Mason University, deans of arts and sciences are using assessment 
data to inform planning, hiring, and resource allocations. 

Examples of change initiated by provosts or presidents are 
rarer. One president in Virginia, in his efforts to restructure 
the institution to meet the need for more high-quality services 
for more people at lower cost, has announced his intention of 
asking programs to justify their existence after five years, and 
he has said that assessment data will be relevant to the case 
they make. 

We have been warned about using assessment information to 
make negative decisions, since raising the stakes may jeopardize 
the objectivity with which faculty approach the task. But it has 
been my observation that departments that do assessment 
thoughtfully — and that are therefore engaged in a process of 
continuous self-ref lection and improvement — are generally not 
the weak ones. Having a good assessment process should be one 
measure of a program's quality* Too, if a unit fails to 
contribute to the development of students over a period of time 
during which it has tried to improve, it is important to know 
that. That is one important data point — in addition to 
centrality to mission, duplication, cost, student demand — that 
should be considered in deciding whether a program should 
continue or not. 

Account ab i 1 i ty 

As I said, assessment has always had another agenda, usually 
described, somewhat misleadingly , as "accountability." I say 
"misleadingly" because, as David Potter said to you two years 
ago, the Virginia General Assembly's interest did not stem from 
mistrust in the quality of higher education. "Their tone," he 
said, "was more on the order of curiosity and competitive pride." 
"Tell us a story about how higher education is doing," they were 
saying. 

The problem is, incremental, piecemeal change doesn't make 
much of a story. Departments use assessment to improve what they 
do, not to describe how well they are educating students. What 
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results is a crazy quilt, not a narrative of how well programs, 
much less whvole institutions, are doing. 

There have been some advantages to this characteristic of 
Virginia's assessment program, especially in the early years when 
we were trying to make it as non-threatening as possible. 
Remember, for instance, the fear that a reporter was going to 
produce a matrix of assessment results? But it has had its 
downside as well. First, it threatened to make assessment 
irrelevant to those who needed to have a bigger picture — to 
policy-makers and legislators, who then might wonder why they 
were funding it, or even to the presidents on your own campuses, 
who have been tempted to cut assessment when budgets became lean. 
But for other reasons the need to tell a story about what higher 
education accomplishes has become more acute. 

Originally, David said, the legislature wanted to hear how 
well Virginia's higher-education system was educating students 
because of competitive pride — the question they were really 
asking was "how are we doing compared to N. Carolina?" Now, the 
need has become more urgent because of the mood towards higher 
education that has been kicked up by the bad fiscal situation, a 
situation that I think we must view as a permanent, structural 
shift in state budget. Last year Peter Ewell described this as a 
"fundamental shift in the relationship between higher education 
and the public," a "new emerging attitude about the role of 
higher education as a public enterprise." 

We are hearing that the public is not willing to raise taxes 
to increase funding for higher education, largely because we have 
not been able to tell them clearly what they have gotten in 
return for their already substantial governmental and individual 
investment in it. At the macro-level, have we given the nation 
economic competitiveness? At the micro-level, do we continue to 
provide students access to the professional classes? Higher 
education is having increasing difficulty in making those claims. 
We have not been able to demonstrate how the state and students 
are better off for their money's having been spent — much less 
whether higher education could have served them more cost- 
effectively. 

So, rather than tamper with the assessment system we have 
that is serving one purpose well and replace it with one which 
will serve two purposes badly, the Council proposes to develop a 
set of indicators to supplement the assessment narratives. Which 
indicators we pick will depend on the audience. 

What do parents and students want to know? As the mother of 
a recent college graduate, I think I know some questions my son 
would have liked the answers to: 

^ Will I graduate? The answer to that might lie in graduation 
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and retention rates, contextualized. 



• Will I get a job? Employers might be surveyed: do the 
students get placed and if so, do they have the skills to do 
the job, and the job after that, and the job that is changed 
beyond recognition? 

• Will I learn something that will permit me to progress, 
personally and professionally, in my life? On alumni 
surveys we might ask, are you happy in your life and work 
and did your college prepare you for your adult life? 

• What will my experience be like? To what degree will I be 
taught by full-time faculty? How large will my classes be? 

• Will my learning add up to anything? 

And let me add a parental question to this list: 

• Are there any adults in charge? How well does the 
institution improve what it does by keeping track of what 
students learn from and think about programs? That is, how 
well is the institution doing assessment? 

The pressure of these questions is going to increase as we 
pass more of the cost of higher education on to students and 
parents, since that engenders, as David Potter said two years 
ago, a " consumer ist mentality" in them. Legislators are 
extremely sensitive to that pressure • In addition, they are 
likely to focus on what Peter called the "production function" of 
higher education — that is, how we will handle increased numbers 
of students with a permanently smaller fiscal base. The question 
they would ask in this context is, are we doing things as 
efficiently and effectively as we could? In order to answer that 
question, the Council has asked its staff to come up with some 
efficiency and productivity indicators • 

We can already demonstrate that institutions are more 
efficient than in previous decades, in that the colleges and 
universities in Virginia are educating 28% more students with 16% 
more faculty and 3% more support employees than they were in 
1980. But that argument is incomplete, if what you want to know 
is how much learning was produced at what cost. We need to bring 
the two kinds of data together to demonstrate that we are 
becoming more efficient without a loss of quality and even an 
improvement in it. Essentially, we need to tell a story about 
return on society ' s strategic investment in higher education. 

Re porting 

The second major question I said I would address today is 
how the reporting relationship between the Council and the now- 
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mature assessment programs should change as a result of the 
maturation of the system* We are looking for reporting 
procedures that meet your needs to maintain the visibility and 
importance of assessment on your campuses while reducing the 
reporting burden to you and the reviewing burden to us. We, like 
you, have got to work smarter and more efficiently if we are to 
survive into the brave new century. 

What I would like to put before you today is a suggestion 
developed by an ad hoc group of assessment coordinators and 
faculty who met in Richmond several weeks ago. The aspects of 
present reporting procedures that they liked were that they keep 
the pressure on on campus, and that participation in the review 
process is a good learning experience for assessment coordinators 
who are reviewers. They liked a common format that permitted 
them to compare what they are doing to what happens on other 
campuses. On the other hand, they found the report writing 
onerous, and some expressed the opinion that the kind of basic 
questions we ask let the more advanced programs off the hook. 

So they ended up recommending an expanded, more open 
process, involving all assessment coordinators, all provosts, and 
more faculty. They recommended a process in which we could ask 
institutions hard questions such as "How do you know your 
general-education program is making a difference to students," 
and "How can you use what you've learned from assessment to 
improve it?" 

The group suggested that rather than submitting 75-page 
assessment reports that are then reviewed by a team and responded 
to by letter or a summary sheet, each campus would submit a 
summary of assessment activities and results to a review team 
made up of assessment coordinators and a provost from another 
campus. Then, the chief academic officer, the assessment 
coordinator, and other personnel of each campus would do a public 
presentation of their institution's assessment program to the 
review team and other interested bystanders. The presentation 
would last 45 minutes to an hour and be followed by questions 
from the team. The team would then, first, give its reactions to 
the program; second, ask for any follow-up information it needed; 
and third, provide advice and assistance to the institution about 
how the program might be strengthened. 

If you don't find this suggestion fundamentally flawed, I 
will take it to the chief academic officers and, presuming a 
favorable reaction, ask several campuses to pilot test the 
process next year. After that, we may change the interim reports 
as well — perhaps replacing them with a series of regional 
meetings at which you could respond to each other's programs. 

In closing, let me thank you for all the good work you have 
done in the past year. Dickens referred to the period of the 
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French ^Revolution as "the best of times, and the worst of times." 
I think we all know how it's been the worst of times. But as in 
any period of major upheaval, these are also the best of times in 
that they call on the commitment and creativity that many of you 
have shown yourselves to have in full measure. I honor and thank 
you for that. 
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